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Shannon's Capacity Theorem is the main concept behind the Theory of Communication. It says 
that if the amount of information contained in a signal is smaller than the channel capacity of a 
physical media of communication, it can be transmitted with arbitrarily small probability of error. 
This theorem is usually applicable to ideal channels of communication in which the information 
to be transmitted does not alter the passive characteristics of the channel that basically tries to 
reproduce the source of information. For an active channel, a network formed by elements that are 
dynamical systems (such as neurons, chaotic or periodic oscillators) , it is unclear if such theorem is 
applicable, once an active channel can adapt to the input of a signal, altering its capacity. To shed 
light into this matter, we show, among other results, how to calculate the information capacity of 
an active channel of communication. Then, we show that the channel capacity depends on whether 
the active channel is self-excitable or not and that, contrary to a current belief, desynchronization 
can provide an environment in which large amounts of information can be transmitted in a channel 
that is self-excitable. An interesting case of a self-excitable active channel is a network of electrically 
connected Hindmarsh-Rose chaotic neurons. 



I. INTRODUCTION 

Given an arbitrary time dependent stimulus that ex- 
ternally excites an active channel, a network formed by 
systems that have some intrinsic dynamics (e.g. oscil- 
lators and neurons), how much information from such 
stimulus can be realized by measuring the time evolu- 
tion of one of the elements of the channel ? The work in 
Ref. [1] shows that 50% of the information about light 
displacements might be lost after being processed by the 
HI neuron, sensitive to image motion around a vertical 
axis, a neuron localized in a small neural network of the 
Chrysomya magacephala fly , the lobula plate. Does that 
mean that the HI neuron has an information capacity 
lower than the information contained in the light stimu- 
lus ? Or does that mean that information is lost due to 
the presence of internal noise ? 

In order to be able to shed light into these questions, 
we need to know how to calculate the information ca- 
pacity of an active channel, and, for practical purposes, 
to understand how an active channel (an active network 
composed by dynamical systems) behaves if it is set up 
to operate with its information capacity. As we shall see, 
there is a parameter route through which the information 
capacity is reached, and this route can be established in 
terms of either the coupling strengths or the level of syn- 
chronism (behavior) among the elements forming the ac- 
tive channel. While information might not always be easy 
to be measured or quantified in experiments, one might 
be able to state about how good is an active channel 
to transmit information by measuring synchronization, a 
phenomenom which is often not only possible to observe 
but also relatively easy to characterize. 

Synchronization is vital for modern methods of digital 



communication that rely on the synchronous operation 
of many subsystems Q. Similarly, transport networks 
depend crucially on the synchronous operation of each 
subnetwork. If one subnetwork gets out of synchrony, the 
whole network might failure to function properly. So, it 
would be intuitive to say that complex systems should 
have subsystems that operate in synchrony for a proper 
functioning. In fact, synchronization between neurons in 
the brain is believed to provide a good environment for 
information transmission. This comes from a fundamen- 
tal hypothesis of neurobiology 3, 4, 5, 6] that synchro- 
nization 0, [1| functionally binds neural networks coding 
the same feature or objects. This hypothesis raised one 
of the most important contemporary debates in neurobi- 
ology, but is still controversial 0, because desynchro- 
nization seems to play an important role in the percep- 
tion of objects as well. 

The analyses are carried out using among others two 
quantities suitable for the treatment of information trans- 
mission in active channels, the channel capacity and the 
system capacity. In short, the channel capacity measures 
the maximum rate with which information is exchanged 
between two elements of the active channel, a path along 
which information can flow in the active channel. On the 
other hand, the system capacity is the maximum of the 
Kolmogorov-Sinai entropy, the total amount of indepen- 
dent information that can be simultaneously transmitted 
between all the pairs of elements of the active channel. 

Among our main results, we show that the channel 
capacity of an active channel depends on whether the ac- 
tive channel is self- excitable or not (see definition in Sec. 
imp . Active channels composed of non self-excitable sys- 
tems (such as Rossler-type oscillators) achieve its maxi- 
mal channel capacity to transmit information whenever 
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its elements are in complete synchrony. On the other 
hand, active channels composed of self-excitable systems 
(such as neurons), achieve its maximal channel capac- 
ity when there is still at least one degree of freedom or 
characteristic oscillation (time-scale) that is out of syn- 
chrony. In the case of active channels formed by spiking- 
bursting neurons (Hindmarsh-Rose electrically coupled), 
the maximal channel capacity to transmit information is 
achieved when the neurons phase synchronize in the slow 
time-scale (bursting) and desynchronize in the fast time- 
scale (spiking). Thus, synchronization in neural networks 
might be two folded. Depending on the type of measure- 
ment made, one can agree or disagree with the binding 
hypothesis. 

Therefore, in this work, wc build a bridge between 
Shannon's Theory of Communication .llj and the The- 
ory of Information in dynamical systems |12l | contributing 
to the development of a nonlinear Theory of Communi- 
cation applied to dynamical systems, shading some light 
in these paradigms of neurobiology. These new ideas, 
concepts, and theoretical approaches unravel the rela- 
tion between stimuli p^ . information capacity, and syn- 
chronization, in a nonlinear media of communication, the 
active channel^ a network formed by elements that are 
dynamical systems. 

II. MUTUAL INFORMATION RATE 

A good communication system as visualized by Shan- 
non comprises a transmitter who transforms a message in 
a signal suitable to be transmitted through the channel 
and a receiver who recovers the message from the signal. 
In analogy to this definition and in order to properly deal 
with information transmission in dynamical systems, we 
define three subspaces in the active channel. The sub- 
space a that generates a suitable message to be trans- 
mitted, regarded as the transmitter (an element of the 
network), the subspace /3 where the message can be re- 
covered, regarded as the receiver (another element of the 
network), and finally the composed subspace (a,/3), con- 
sidered to form one communication channel, an union 
of the subspaces a and /3, also a subspace of the active 
channel. So, both the transmitter and the receiver be- 
long to the communication channel. Herein, the more is 
the information exchanged between the receiver and the 
transmitter, the more information about the transmit- 
ter's trajectory can be realized in the receiver. The tra- 
jectory of the transmitter (receiver) represents the evo- 
lution in time of the transmitter's (receiver's) position in 
the subspaces a (/3). 

According to Shannon, the amount of information that 
can be measured in the receiver, x^^\ about the trans- 
mitter of information, a;*^"^ , is given by 

I{a,P) ^ H^"H H^^^ - H'^"-'^\ (1) 

also known as mutual information between the transmit- 
ter and the receiver. is the information produced 



by the transmitter, _ff('^\ the one produced (or measured 
in) by the receiver, and i?^"''^) the one produced in the 
composed subspace, also known as the joint entropy be- 
tween the receiver and the transmitter. To calculate the 
terms in Eq. ([T]) for systems where events in the future 
are connected to events in the past (systems with corre- 
lation), one usually needs to coarse grain the domain of 
the subspaces a and (3 into n equal size-e [14] intervals 
ai and /Sj, with i, j = 1, . . . , n (n = 1/e), being that Pj^T'* 

(Pm^) represents the probability of an event, e.g. a tra- 
jectory point in the subspace a {13) visiting a sequence 
of L intervals. The trajectory remains a time r in an 
interval e. If one is working with maps, r=l. The term 

P^T'^^ represents the probability of a composed event, 
e.g. a trajectory point visiting an itinerary following a 
sequence of L areas, each area delimited by the intervals 
ai and fij, as represented in Fig. [TJ Then, in Eq. ([1]), 

^^"^=- E,„ Pi^^ InP/n \ H^^)^- E,„ Pif^ \^P^\ and 
^(a,/3)^_^^^^p^(«./3)lj^p^a,/3)^ whcrc we have taken the 
limit of (e, t) — > 0, L — > oo. 

Notice that all the terms in Eq. ([1]) tend to infinity 
as (e, r) — > 0, L — !■ oo. So, it is advantageous to work 
with terms a = H/(tL) that measure the amount of 
information per time unit, which are finite quantities in 
the active channel. So, we rewrite Eq. ([T]) as 

/(a,^)= lim tL((t(") -(t("'«) (2) 

L^oo,(r,e)^0 

/c(a,/?) = I{a, /3)/{tL) is the mutual information 
rate (MIR) between the transmitter (a) and the re- 
ceiver (/3). Based on the results of the term d^""'') 
is the Kolmogorov-Sinai (KS) entropy of the tra- 
jectory in the subspace (a,/?) regarded as H^^f\ 
Imagine that the receiver has a finite physical coupling 
with the transmitter. From Takens theorem [l3|, the en- 
tropy of a trajectory calculated in a subspace, e.g. (a), 
should provide the entropy of the trajectory in the whole 
space (a,/3), which leads to that a^°'^=a^f^^=a''°''^\ and 

therefore, /c(a, P)=h'^s^^ ■ Independent on the couphng 
strength and on the synchronization level between the re- 
ceiver and the transmitter, one arrives that the MIR is 
constant and given by H^x 's ^ ■ Naturally, in order for one 
to gain such an amount of information rate, one might 
have to realize an infinite number of observations in the 
receiver's trajectory and one has to have access to a good 
trajectory projection (subspace). But, in communica- 
tion, it is desirable that information about the transmit- 
ter can be "instantaneously" realized in the receiver. In 
addition, measurements performed in the subspace (3 of 
the receiver do not necessarely contain all the informa- 
tion content produced by either the whole active channel 
or a communication channel. For these reasons, we in- 
troduce a consistent definition for the MIR between two 
subspaces (elements) in a network 

Jc(a, P) = I A(") I + ) I At'') I - H^^f'^ (3) 



3 



where A^") and A^^^ are the Lyapunov exponents of 
the trajectories in the subspaces a and /3, respectively, 
which measures how nearby trajectories exponentiaUy 
diverge in these subspaces, D^"^ and D^^^ are the in- 
formation dimension of the trajectory in these subspaces 
[see Appendix [A] . and |.| is the modulus operation p^ . 
By doing that, we assume that the first two terms 
(i:)(")|A(")|+D(^)|A('')|) in the right side of Eq. © mea- 
sure the information produced by both the receiver and 
the transmitter as if they were uncoupled, i.e., uncorre- 
lated "random" variables (no phase-space reconstruction 
is employed in the measurable data from the sub- 
spaces a and /?), and the last term (Ji'^^s^) provides the 
dependence between them. These are actually the ba- 
sic assumptions behind the definition of mutual informa- 
tion provided by Shannon to random variables being 
transmitted through a noisy channel. For more details 
about how to analytically calculate the terms A'^ \ see 
Appendixes El and |B1 



III. INFORMATION CAPACITY, 
EXCITABILITY, AND SUSCEPTIBILITY 

In order to study the way information is transmitted 
in active networks, we introduce quantities and termi- 
nologies that assist us to better present our ideas and 
approaches. 

An active channel is an active network constructed 
using Q elements that have some intrinsic dynamics and 
can be described by classical dynamical systems, such as 
chaotic oscillators, neurons, phase oscillators, and so on. 
Every pair of elements forms a communication chan- 
nel and the rate with which information is exchanged 
between these elements, a transmitter and a receiver, 
is given by the mutual information rate (MIR) between 
them. 

The channel capacity, Cc, of a communication chan- 
nel is defined as the maximum of the MIR for this com- 
munication channel formed by a pair of elements, the 
receiver and the transmitter, with respect to many pos- 
sible coupling strengths among the elements, for a given 
network topology. Thus, the channel capacity is the max- 
imal possible amount of information that two elements 
within the network with a given topology can exchange, 
a local measure that quantifies the point-to-point rate 
with which information is being transmitted. Notice that 
a communication channel is a subset of an active network. 

The system capacity, Cs, of an active network com- 
posed by Q elements is defined as the maximum of the 
Kolmogorov-Sinai (KS) entropy, Hks, of the whole ac- 
tive network {Hks > h'ks^)^ with respect to many pos- 
sible coupling strengths among the elements, for a given 
network topology. The Kolmogorov-Sinai entropy offers 
an appropriate way of obtaining the entropy production 
of a dynamical system. Here, it provides a global measure 
of how much information can be simultaneously transmit- 



ted among all communication channels. Therefore, Cs 
bounds Cc(q!,/3) as well as the KS-entropy, Hks, of an 
active network, calculated for a given coupling strength, 
bounds the MIR between two elements, Ic{oi,(3), calcu- 
lated for the same coupling strength. Thus, 

Cc{a,l3) < Cs 

Icia,P) < Hks (4) 

An active channel is said to be self-excitable (not 
self-excitable) when Cc > H^iPs (when Cc < ^ks)' 
with h'^k\ representing the KS entropy of one of the Q 
elements forming the active channel, before they are cou- 
pled. Analogously, we can also define self-excitability in 
terms of the channel capacity. For a self-excitable chan- 
nel, it is true that Cs/Q > H^^\.. Thus, in a self-excitable 
active channel Hks increases as the coupling strength 
among the elements increases. 

An active channel is said to be susceptible if Cc > 
ijj^g and not susceptible if Cc < H^ks^ where -ffj^g 
represents the KS entropy of the uncoupled receiver. So, 
a susceptible channel does not resist the action of the 
stimulus provided by the transmitter or the dynamical 
alteration caused by the coupling configuration in the 
active channel. These alterations might produce also a 
self-excitable channel. It is to be expected that a self- 
excitable channel is also a susceptible one. 

IV. THE CHAOTIC CHANNEL 

Here, we analyze how a source of information can be 
transmitted through a channel that stretches the ampli- 
tude of the information signal [l^, [l^]. The Lyapunov 
exponent of the receiver, A'^, is always positive even if 
there is no coupling between the transmitter and the re- 
ceiver. Part of the information transmitted might be lost 
due to the presence of chaos in the channel. We assume 
that a general source of information can be modeled by 
a chaotic system. 

A model of the chaotic channel is given by two one- 
dimensional chaotic maps bidirectionally coupled [2l|, 

= 2x(f ) -h 2c(4"^ - xiP^),mod{l) (5) 
x^:l = 24") + 2c(4'3) - 4")), mod(l) (6) 

where the subspace of Eq. © is regarded as the receiver 
and Eq. ([S]), the transmitter. In Fig. [1] we represent 
the map trajectory for c = 0.24, the coupling strength. 
This map has two Lyapunov exponents: Ai = In (2) and 
A2 = In (2 — 4c), Ai measures the exponential divergence 
of nearby trajectories in the direction of the synchro- 
nization manifold defined as x*^") — x'^^'^—Q, and A2 the 
exponential divergence of nearby trajectories in the di- 
rection transversal to the synchronization manifold. The 
exponents A(") and A^'') that measure the exponential di- 
vergence of trajectories along the subspaces a and (3 are 
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FIG. 1: The subspace a of the transmitter who generates 
the information, the subspace /3, where information about the 
transmitter can be realized, and the composed subspace (a, /3) 
that represents a two-dimensional active channel of commu- 
nication. 



equal to max (Ai, A2)=Ai, and therefore, A*^"-' = A^'^'^Ai, 
since the maps have equal parameters [see Sec. IVIIj . 
One can also arrive to this result by noting that the Lya- 
punov exponent of a typical ID projection of a 2D chaotic 
set (virith two positive Lyapunov exponents) is given by 
the Largest exponent. Since the trajectories in the sub- 
spaces a and /3 have uniform probability distribution and 
the information dimension of the trajectory in the com- 
posed subspace is 151=2, a one dimensional projection of 
it should provide D'f^=D^^ = l (see Appendixes VK\ and 
IB|) . Using a result by Pesin [l^, the Kolmogorov-Sinai 
entropy of a chaotic system is the summation of all the 
positive Lyapunov exponents, Hks — ^- For a two- 
dimensional channel, H^^g^ equals the entropy of the 
whole channel, Hks- So, Hks = Ai -|- A2, if A2 > 0, or 
Hks — ^1, otherwise. Therefore, we arrive that the rate 
with which information can be retrieved in the receiver 
about the stimulus generated in the transmitter is given 
by /c = Ai — A2, if A2 > 0, or Ic = Ai, otherwise. So, 
Cc = In (2) and Cs — 2 In (2). An increase in the cou- 
pling leads to an increase in the MIR and a decrease in 
Hks- 

To relate the MIR with the synchronization level of 
this chaotic channel, we make a convenient coordinate 
transformation into new variables x" and x'^ (see Ap- 
pendix [X| such that the exponential divergence on a:-*" 
is minimal {x^ is oriented along the contracting direc- 
tion) and x" is orthogonal to (x" is oriented along 
the expanding direction). Such a transformation, for the 

systems to be treated here, is xl"'''' " =xi"'' -|- xi^"* and 



Xri =Xn — x„ , with the synchronization manifold 
given by x^=Q. The mapping in this new coordinate 
generates the conditional Lyapunov exponents A" ~ Ai 
and A"*" = A2. In contrast to the conditional expo- 
nents defined in Ref. (2^ . the obtained exponents pro- 
duce physically consistent quantities (ergodic invariants) 
[23 | even for situations when complete synchronization 
is absent. The transformed equations of motion in these 
new variables provide not only the same Lyapunov ex- 
ponents but also, for this example, the same KS entropy 
{Hks = All + A^, if A^ > 0, or Hks = AH, otherwise) 
of the original equations, and advantageously supply us 
with a way to understand the synchronization level be- 
tween the two subsystems. A" > A^, and we recover 
the conjecture of Ref. 0, /c = AH - A-^, if A-^ > or 
Ic = a" , otherwise. This conjecture provides an easy way 
of solving Eq. ([3]) for an active channel linking informa- 
tion to synchronization. The more synchronization, the 
smaller is x"*", and therefore, nearby initial conditions in 
this variable will diverge exponentially in a smaller rate, 
i.e., the conditional exponent associated with this vari- 
able, A"*", is smaller. If A-*" is very small. A" can be asso- 
ciated with the amount of information production of the 
synchronous trajectories (between the transmitter and 
receiver), otherwise, it is associated with the excitation 
of the channel. The more excitation in the channel the 
larger is A". So, to achieve larger amounts of information 
transmission it is required that either the excitation or 
the synchronization level are large, or both. In this chan- 
nel, as we increase the coupling, Hks decreases due to an 
increase in the synchronization level, which leads to an 
increase in the MIR. Cc = Jiim (Hks) 7^ Cs is achieved 
for a configuration when the synchronization is maximal. 
Therefore, the larger the coupling is, the less information 
the whole active channel produces (KS-entropy, Hks), 
but the larger the MIR between a receiver and a trans- 
miter is, which means that the more information about 
the transmitter can be measured in the receiver. More 
synchronization implies more information transmission. 
This channel is not self-excitable and since Cc = H^iPs' 
it is not susceptible, because its capacity is limited by the 
capacity of the receiver to generate information 

In order to calculate the MIR of a communication 
channel in a large chaotic active channel, we need to use 
the coordinate transformation x" and x^. This trans- 
formation enables one to calculate the MIR between two 
subsystems as if they were detached from the active chan- 
nel. Imagine an active channel formed by Q fully bidi- 
rectionally coupled chaotic systems: 

Q 

x^, = 2x(f" ) + 2c(x« - x« )mod(l) (7) 

i=l 

with j = [1, . . . , Q]. Now, we can define [Q x (Q — l)]/2 
pairs of subspaces. For instance, the pair of subspaces 
formed by the subsystem x^^^ and the subsystem x'^^\ 

with Xn^^"=xl"^^ -\-Xn'^ and Xn ^^"""^xl,^^ — xi^^ Any pair 
of subspaces produces the same conditional exponents 



All = In (2) and A-^ = ln|2(l-Qc)| [see Appendix E]. 
In fact, this system produces one Lyapunov exponent 
A = A'I and {Q — 1) equal others A = A^, and so, our 
defined conditional exponents can be related to the Lya- 
punov exponents even in higher-dimensional systems. So, 
the MIR [see Eq. (|B1|) ] between any two subsystems 
a;('') and a:;^') is Ic{x^''\ x'-'-^) = -ln(l - Qc) bits per it- 
eration of the mapping, for c < 1/(2(5) {^'^ 0). If 
there is no coupling (c—O), then Ic{x^''\ a;*^'-') = and no 
information is exchanged between both subspaces. For 
c < 1/(2(5), the larger is c the more synchronous a trans- 
mitter, say x^''\ is with a receiver, say and the 
more information is exchanged. The channel capacity for 
all communication channels is achieved for c > 1/(2(5), 
when J(7(a:('^\ x^'^) = ln(2) = mm{HKs), and the net- 
work completely synchronizes (A-*- < 0). This type of 
active channel is not self- excitable. Notice that the intro- 
duction of one more element into this channel (13i] does 
not alter Cc- It is also not susceptible. 



V. THE PERIODIC CHANNEL 

The purpose of the present work is to describe how in- 
formation is transmitted via an active media, a network 
formed by dynamical systems. There are three possi- 
ble asymptotic stable behaviors for an autonomous dy- 
namical system: chaotic, periodic, or quasi-periodic. A 
quasi-periodic behavior can be usually replaced by either 
a chaotic or a periodic one, by an arbitrary perturba- 
tion. For that reason, we neglect such a state and focus 
the attention on active channels that are either chaotic 
or periodic. 

The purpose of the present section is dedicated to an- 
alyze how a source of information can be transmitted 
through active channels that are periodic, channels that 
squeeze the amplitude of the information signal. More 
specifically, channels whose receiver behaves in a peri- 
odic fashion (its Lyapunov exponent, A'^, is negative). 

It is to be expected that in the periodic channel a frac- 
tal set appears, when Ai < IA2I (assuming a bidimen- 
sional channel). This clearly imposes severe limits for 
the recovery of information in the receiver. The periodic 
channel can be imagined as a filter. As shown in [2^ . 
chaotic signals being transmitted through filters might 
produce an output with higher dimension due to the ap- 
pearance of a fractal set. To see that we study the gener- 
alized baker's map [25|, shown in Figs, [^a-b) and in Fig. 
131 All the information produced in the transmitter x^"' 
is transfered to the receiver x^^\ but with a time delay. 
To make it more clear, note that in Fig. [2}d, by recogniz- 
ing if the received signal is either smaller or larger than 
b at the iteration time n -\- 1, one is able to know if the 
position of the transmitter was lower or higher than a, 
at the iteration time n. By looking the received signal at 
higher resolution one is able to predict with higher res- 
olution the position of the transmitter farer away in the 
past. 
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a b 




X'J 1 b jl^l^ 1 

FIG. 2: a. The baker's map squeezes the rectangle above a in 
a into the right rectangle in b and it squeezes the rectangle 
bellow a in a into the left rectangle in b. Both rectangles in 
a are stretched in the horizontal direction. From a to b, we 
represent one application of the generalized baker's map. 

In order to calculate the MIR between the transmitter 
and the receiver using Eq. ^ note that A^"^ =a In [1/a] + 

(1-a) In [1/(1 -a)], D^"^ = l, A('3)=aln (6) + (l-a) ln(6), 
and Df^^X^^yiX^f'^l (with d[^^ < 1) and A("'« ^ 
Hks = A(") + A^''). If L>f ^ < 1, Af/^) < 0, and A^") < 
A^'^-'l, and a fractal set takes place. As demonstrated in 
Ref. [2^, the information content of this fractal set is 
7?f^|A('5)| [23. Thus, Ic^X^'^^ + Df^X^fi^- Hks. and 
we arrive at /c=A'-"'' per iteration, since Hks = 
and \X^P^\Df^ = A("). Hks = A^") because the frac- 
tal set does not contribute to the KS entropy of the full 
chaotic map. So, Cc depends on the amount of informa- 
tion produced in the transmitter, a typical characteristic 
of a susceptible channel. Unlike the chaotic channel that 
is robust to small noise intensities HI /, in the periodic 
channel Cc might be extreme sensitive to noise of even 
arbitrary amplitudes. 

In active channels, the receiver might influence the 
transmitter behavior. We can imagine a bidirectional 
coupling scheme for which a periodic uncoupled receiver 
might make an uncoupled chaotic transmitter to behave 
also periodically, after the coupling is switched on. This 
type of periodic channel is thus not susceptible to adapt 
to stimuli, and Cc—0. 

The periodic channel might be relevant to understand 
the role of subthreshold oscillations in the processing of 
information. These oscillations are observed in the brain, 
in particular to regions associated with motor reaction 
and learning like the Inferior Olive [2^ . They appear 
because groups of neurons interact in such a way that 
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FIG. 3: The asymptotic solution of the map in Fig. [21 formed 
by a series of vertical strips forming a fractal set in the hori- 
zontal direction. 

the potential in the neurons membrane is not sufficient 
to induce a spike. Usually, the oscillations arc reduced 
to a limit cycle, a periodic behavior. 

VI. THE NEURON CHANNEL 

We illustrate our ideas in a relevant type of ex- 
citable active channel, the chaotic neural channel [s^, 
[Sll . [3^ . formed by a network of electrically connected 
Hindmarsh-Rose (HR) neuron models [HI. This net- 
work possesses both characteristics of the periodic and 
chaotic channels, since it has both positive and nega- 
tive Lyapunov exponents. A fractal set that occupies a 
small portion of the phase space coexists with a chaotic 
set that occupies most of the phase space. Due to the 
negative exponents, the dynamics is strongly compressed 
along the stable directions, the stable manifolds. The re- 
sult is that the observable dynamics of the neurons lies 
along the unstable manifolds, and thus, the negative ex- 
ponents do not contribute to the recovered information. 
This is a consequence of the Sinai-Ruelle-Bowen (SRB) 
assumption. 

Each pair of neurons can be treated as an active chan- 
nel of communication, one neuron ak performing the 
transmitter task and the other f3i the receiver. In Ref. 
[20j . we have stated that the amount of information 
in one chaotic channel should be always smaller than 
the information produced by the network, Hks- Thus, 
Ic{ak,Pi) < Hks [see Eq. (j4])]. By working with large 
networks, composed of many elements, we should expect 
that the same information travels simultaneously along 
many different channels. This property, often desired 
in networks, makes it a reliable medium for informa- 



tion transmission because it introduces in the network 
a large amount of redundancy, which results into a sum 
of all the MIR in the communication channels larger than 
Hks- Even if one or many channels are blocked, the in- 
formation still finds its destination. So, to treat networks 
composed by Q chaotic systems, we expect that 

(Ic) < Hks- (8) 

where (Ic) ~ J2k i ^cio^k, Pi)/ P represents the average 
amount of MIR of the whole network, P is the number of 
communication channels given by P = [Q{Q — l)]/2, and 
Q is the number of neurons. For the neuron channel, we 
consider that Cc represents the maximal of (Ic) for many 
coupling configurations. The average amount of redun- 
dancy in the network is defined to be (R) = -jf^, and 
thus, if the network is completely synchronized (i?) =1, 
and if the network is completely uncoupled (i?)=0. 

Wc consider a network composed of Q=4 bidircction- 
ally fully coupled neurons: 

— Hi ^ -^i ^ li ^ ^ ^ '^j^i.'^j '^0 

j 

= l-5xf - yi (9) 
Zi — ^rzi + 4r(xi + 1.6) 

The parameter that modulates the slow dynamics is set 
to r=0.005, such that each neuron is chaotic, i and j, 
with j ^ i assume values within the set [1, . . . ,Q]. ak 
represents the subsystem of the variables {xk,yk, Zk) and 
Pi represents the subsystem of the variables {xi,yi,zi), 
where fc=[l, . . . , Q — 1] and l=[k -I- 1, . . . , Q]. Aji = Aij 
[m is the strength of the electrical coupling between the 
neurons represented by aj and a^. The external stim- 
ulus /i, in «!, is set to be equal to Ii = 3.25-(5/, and 
then, h=h+5I, h=Ii-5I, h=h+5I, with 51 = 0.00001. 
Initial conditions are a;=-1.3078+ry, ?;=-7. 3218-1-77, and 
z=3.3530-f 7y, where 77 is an uniform random number 
within [0,0.001]. 

Four synchronization phenomena are relevant to be 
considered [see Appendixes El and [C] . Bursting phase 
synchronization (BPS), when at least one pair of neu- 
rons is phase synchronous in the bursts, partial phase 
synchronization (PPS) when at least one pair of neu- 
rons is phase synchronous in the bursts and in the spikes, 
phase synchronization (PS), when all the pairs of neurons 
are phase synchronous in the bursts and in the spikes, 
and complete synchronization (CS). An evidence for the 
presence of bursting or spiking phase synchronization is 
found if the condition max(AAf")/P < 1 is satisfied, 
where = Y.k i - and NJ^ represents the 

number of spikes/bursts in Uk, at the time the neuron ai 
suffered its n-th spike/burst. This condition is threshold 
dependent, but it will be employed here for the purposes 
of illustration. 

This example, illustrated by Fig. [H shows three fun- 
damental characteristics of an active channel: 

(i) Excitation enhances Hks o-nd the MIR of the com- 
munication channels. With no coupling, the rate of in- 
formation production in each neuron is approximately 
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H^^g = 0.014 and (Jc) is null. For a coupling strength of 
^ki,ik > 0.01, each pair of neuron exchanges (in average) 
information with each other in a rate larger than the indi- 
vidual rate with no coupling. So, an increase in the cou- 
pling strength is simultaneously followed by an increase 



in both Hks and the rate of information production of 
each individual neuron, resulting in an increase of (Ic), 
meaning also an increase in the MIR of the communica- 
tion channels, a typical characteristic of a self-excitable 
channel. 




FIG. 4: [Color online] a, Circles, squares, and diamonds show Hks, (Ic), and (_R*) = {J?)/10 for a network of 4 fully connected 
neurons. The necessary conditions for the proper calculation of (Ic) are not satisfied for coupling strengths that produce 
stronger types of PS {Aki = [0.15,0.25]), see Appendix [K\ There, (Ic) should be estimated by a finite time MIR, indicated 
by {Ic)f- For this coupling strength interval, we set {R*)=l. b. Circles and squares represent the maximal average spiking 
and bursting difference, max (AA*'"/P), after n=200 bursts, in a log vertical axis. Values of max(AA^"/P) smaller than the 
dashed line are an evidence that there is PS. The upper arrows in a indicate the coupling strength intervals for which we find 
BPS {Aki = [0.1,0.23[), PPS (Afci=[0.23,0.245[), PS (^^, = [0.245, 0.25[), and CS {Am > 0.25). To obtain the information in 
units of bits we divide the related equations by In (2). Ic is calculated using Eq. (|Bip . which produces similar values to the 
ones obtained by calculating the MIR from the entropy and joint entropy of symbolic sequences generated from the trajectory 
of pair of neurons. 



(ii) Synchronization does not necessarily mean high 
levels of information transmission. When the network 
reaches its system capacity {Aki «0.08), the spikes and 
the bursts are highly desynchronous [Fig. |3|d] by usual 
definitions of phase synchronization [see Appendix [C] . 
but both (Ic) and the redundancy {R) are high. At 
this point, we have to remember that MIR means the 



amount of excitation minus the amount of desynchro- 
nization. The amount of excitation is of the order of 
the maximal Lyapunov exponent of the network, which 
is large, since the network is excited, much larger than 
the amount of desynchronization. On the other hand, 
for Aki > 0.23, when the neurons phase (PPS or PS) or 
completely synchronize (^fci > 0.23), {Ic) abruptly drops 
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approaching the low value of Hks, much lower than C5, 
as if the whole network were formed by one single neu- 
ron. The redundancy is high but few information can be 
transmitted in the network. 

(in) BPS provides an ideal environment for informa- 
tion transmission. When BPS is present, (/c) and the 
redundancy are high. Hks — (Ic)- That suggests that 
BPS plays an important role in the reliable exchange of 
information that demands rapid responses and a large 
amount of information transmission. Each neuron main- 
tains a high level of independent activity (given by the 
desynchronous spikes) and simultaneously a moderate 
level of synchrony (synchronism in the bursts) that al- 
lows a neuron to "talk" to another. These characteristics 
are usually desirable in sensory neurons [33| and in the 
ones responsible for motor reaction processes. 

VII. ACTIVE CHANNELS FORMED BY 
NON-EQUAL ELEMENTS 

Here, we briefly describe the dependence of the MIR on 
the parameter mismatches between the elements forming 
an active channel. For such a case, A*^"^ and A'-''-' typi- 
cally differ, in Eq. As a consequence, the channel 
capacity is lower than when the parameters do not mis- 
match and Eq. (jBip should be considered as an upper 
bound (it might overestimate the real value) for the MIR 
between the subspaces ak and ai [35l|. Also, a parameter 
mismatch might enhance the value of the MIR if the cou- 
pling is kept constant while the parameters are changed. 
For simplicity, let us represent an active channel formed 
by two coupled unidimensional maps by {xl^_^i, x^J^^-^). 
Thus, A("), which gives how nearby trajectories expo- 
nentially diverge along the subspace a, can be calculated 

from — -I Ygy-, the sum of the terms in the row of 

the Jacobian with respect to subsystem a, and A^*^^ from 

OS*"' dx'-'^'i 

— T^f "I TFT) the sum of the terms in the row of the 

dxn dx)^ 

Jacobian with respect to subsystem /3. 

As an illustration, imagine the following chaotic chan- 
nel: 

xii'l, = {a- e)x(p + 2c(4") - a;(f )), mod(l) 

xi% = axi") + 2c(4'') - 4"^),mod(l) (10) 

with a > 1 and e is the parameter mismatch. Then, 
we arrive at A(")=ln(a), A('3)=ln (|a - e|), Ai=AII = 
In (l/ii -I- ^2!), and A2= A^= In (|/xi - fi2\), with ^1 = 
a — 2c — e/2 and /12 = 2^ (c^ + 6^/16). If there is no cou- 
pling (c=0), and even if e 7^ 0, /c(a, = 0. To see that, 
we assume e — >0, which lead us to Ai = In (|a — e/2|) and 
A2 = In (|a — 4c — e/2|), and then we use the expansion 
In (a — e)=ln (a) — e/2, in the exponents. The larger is e, 
the smaller is the channel capacity, Cp, which is reached 
for parameters that lead to A2 = (a — 4c — e/2 = 1), 
when the maps, although not completely synchronous. 



have close trajectories. If A2 > (c < — 1/4 — e/8-l- a/4), 
then /c(c, e > 0) > /c(c, e — 0), which means that a 
parameter mismatch can enhance the MIR of the chan- 
nel, more precisely by a maximal amount of e/4. That 
means that parameter mismatches for sufficiently small 
coupling strengths enhance the synchronization level in 
an active channel. Notice that Eq. (jBip is indeed an 
upper bound for the MIR, being that the difference be- 
tween Ic obtained from Eq. (|B1|) and the one from Eq. 
^ is of the order of e^, for small e and c, and for larger 
c it is smaller than e/4. If e=0, then A(")=A(^)=ln (a), 
which results in Cc=ln(a), the maximal possible value 
for the MIR. Finally, the system capacity, Cg = 2 In (a) , 
is reached when c=0 and e = [s^. This channel is not 
susceptible for e = 0. If e ^ 0, it becomes susceptible, 

since Cc > ^^^5 = l^^d'^^^D- However, independently 
on e, this channel is not self-excitable. 

For larger HR neural networks (up to 50 neurons) de- 
scribed by Eq. the system capacity is reached for 
a network of equal neurons, but with a non-zero cou- 
pling strength. We find that Cs is reached for a Small 
World network geometry [s^ . being that Cs increases 
linearly with the number of neurons, Q, by Cs —13.75Q 
bits/burst. The larger Q is, the smaller the coupling 
strengths, which are considered to be equal. Also, the av- 
erage number of connections, 7, that each neuron receives 
scales linearly with the number of neurons as 7 cx 0.5Q. 

Networks formed by HR neurons connected to their 
nearest neighbors, forming a ring, can be regarded as 
models for the small networks of electrically connected 
neurons found in the Inferior Olive [38| that regulates 
the transmission of information between the cerebellum 
and the cortex, and is responsible for motor control and 
learning. In this type of network, we find that the system 
capacity increases linearly with the number of neurons by 
Cs —9.5N bits/burst, being achieved always for the same 
small coupling strength ( 0.08). Therefore, the capac- 
ity does not depend on the coupling strength. Network 
configurations for which the system capacity is reached 
operate also with a large MIR in each communication 
channel. This is an optimal configuration for networks 
found in the Inferior Olive that demand large amounts of 
information transmission for an efficient cerebellar learn- 
ing. Naturally, we do not expect that the neural networks 
found in the Inferior Olive are formed by equal neurons. 
So, the calculated channel and system capacities should 
be interpreted as an upper bound for these quantities in 
realistic models of the networks found in there. 



VIII. THE MUTUAL INFORMATION RATE 
BETWEEN OTHER SUBSPACES 

In this work, we are mainly interested in calculating the 
MIR between the subspaces a and /3 [Eqs. ([3]) and (|B1|) ]. 
That means that we are mainly interested in knowing 
the rate of information that can be realized from a trans- 
mitter (one element of the active channel) by measuring 
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the signal of a receiver (another element of the active 
channel) . 

However, it is of general interest to learn how to cal- 
culate the MIR between groups of elements or between 
different subspaces of the active channel. For example, 
in many experimental situations, one cannot obtain the 
signal of an isolated element but rather an average field, 
or average quantity, such as the quantity x" , which can 
be imagined as an average field (xjl; = + xi) between 
two elements. 

As briefly described here, in fact, we can also calculate 
the MIR between other subspaces. As an illustration, 
we consider the calculation of the MIR between the sub- 
spaces defined by the coordinate transformations and 
x-^, regarded as Ic{x\x-^). 

Typically, we should expect that Icix^x^) ^ 
Ic{a,l3), and therefore, the MIR is coordinate depen- 
dent. While Icia,f3) measures the rate with which in- 
formation about the transmitter a can be realized by 
observing the receiver (3, Ic{x\ x^) measures the rate 
with which information about a;'' can be realized by ob- 
serving x-^. Naturally, at a situation when CS takes 
place, nothing about can be said by observing x^, 
since x^=0, and therefore, Ic{x\x^)—0. So, to know 
from /c (a; 11,5;^) if one element exchanges high amounts 
of information with another element, not only Ic{x^^ , x-^) 
should be low, but also A""". 

Notice that if the network is composed by elements 
with a linear dynamics (or piecewise linear for continuous 
dynamics), we can always find a linear transformation 
from which Ic{x^^,x-^) can be calculated from the state 
variables x, or Ic{ct, P) from the transformed variables 
and x^ . Usually, one would calculate /c(xll,af^) using 
the variables a?" and a?^ and Ic{oi, P) using the variables 
a?" and x^ . 

Therefore, even if a measurement cannot provide the 
signal of isolated elements in an active network, there 
might be situations in which we can still calculate the 
mutual information rate exchanged between two isolate 
elements from the measurements of averages. 



IX. THE TIME DEPENDENT (NOISY) ACTIVE 
CHANNEL 

If all the elements forming the active channel suffer the 
influence of the same time dependent stimulus (or noise) 
and both the stimulus is completely uncorrelated with re- 
spect to the variables x-^ and S"" and also af-*- ^ [such 
that the Jacobian in Eq. (jAl[) is approximately block 
diagonal], then both exponents A-^ and A", calculated for 
the autonomous channel, are not modified by the intro- 
duction of the stimulus. Also, if the largest Lyapunov 
exponent of the channel is not affected by the introduc- 
tion of the stimulus. A" is also not modified. Thus, if the 
previous conditions are satisfied, Eq. (jBl|) calculated for 
the autonomous channel gives the upper bound for the 
mutual information of the non-autonomous channel. 



The periodic channel fully preserves the transmitted 
information, however it might transform it in a fractal 
set which is vulnerable to noise. So, the information rate 
recovered in the receiver can be sensitively dependent on 
the noise level in the channel. Chaotic channels tend to 
destroy part of the information transmitted, even with- 
out the presence of noise (X) > 0)- However, they 
might offer a nice way to deal with additive noise. As 
shown in Ref. [28j . Gaussian noise with small variance 
added to a chaotic trajectory can be completely filtered 
out. 

The action of more general types of time dependent 
stimulus that alters the dynamics of the active channel 
still needs better clarification. 



X. CONCLUSION 

An active channel is an active network composed by 
dynamical systems. Every pair of elements forms a com- 
munication channel and the rate with which information 
is exchanged between two elements, a transmitter and a 
receiver, is given by the mutual information rate (MIR) 
between them. The maximum rate of information that 
can be transmitted in a communication channel of an 
active channel is regarded as the channel capacity, Cc, 
and the maximum rate of information produced in the 
whole active channel is regarded as system capacity, the 
maximum of the Kolmogorov-Sinai entropy, HkSi of the 
active channel. All these maximums are calculated with 
respect to many possible coupling strengths among the 
elements, for a given network topology. 

We can organize the active channels in two main 
classes. Periodic or chaotic. A chaotic (periodic) channel 
is composed by a receiver that behaves in a chaotic (pe- 
riodic) fashion, for long time intervals. Chaotic channels 
formed by HR neuron networks are self-excitable, which 
means that the channel capacity, Cc, is larger than H^^g, 
the Kolmogorov-Sinai entropy of all the elements forming 
the active channel when they are uncoupled. In a self- 
excitable channel, a transmitter and a receiver (together 
with all the other elements forming the channel) mutu- 
ally increase their capacity for information production, 
leading to an increase in their channel capacity. So, the 
introduction of stimuli in a self-excitable active channel 
might increase its channel capacity for information trans- 
mission. Not all chaotic channels present self-excitability. 
For example, we have not verified such a property in ac- 
tive channels formed by linearly coupled Rossler oscilla- 
tors or by linearly coupled Chua's circuit (while in the 
Double Scroll attractor regime) . It is to be expected that 
a periodic channel is non self-excitable. 

More synchronization results in an increase of the MIR 
between two elements in an active channel, regarded as 
transmitter and receiver, if as the transmitter becomes 
more synchronous with the receiver, simultaneously also 
the Kolmogorov-Sinai entropy decreases, meaning that 
synchronization is accompanied by a reduction of the ex- 
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citation in the channel. This situation is to be expected 
in non self-excitable channels. In self-excitable channels, 
a large amount of information transmission can be ob- 
tained when the bursts are phase synchronous while the 
spikes are highly desynchronous. 

Periodic channels might allow the complete transmis- 
sion of the information signal provided by the transmit- 
ter. On the other hand, the transmitted information in 
a chaotic channel might be lost due to the presence of 
non-synchronous chaotic trajectories, if the transmitter 
is weakly coupled to the receiver. However, while a pe- 
riodic channel might be very sensible to the presence of 
even arbitrarily small noise intensities, chaotic channels 
might be robust. 

APPENDIX A: CONDITIONAL EXPONENTS 
AND COMPLETE SYNCHRONIZATION (CS) 

Assume Xk to describe the state variables of subsystem 
k. The parallel and perpendicular subspaces are defined 



to be a transformation in the variables of the subsystems 
that maximize the calculated mutual information. For 
the cases here studied, the parallel subspace between uk 
and a; is defined as x\i = Xk + xi, and the transversal 
subspace is defined as a?^; ^ Xk — xi. For a network of 
Q elements, k — [1,Q — 1] and / = [k + 1,(5]. Writing 
the equations of motion in these new variables, one can 
separate the transformed equations into subsystems that 
contain only terms of that subsystem. So, a network of 
Q elements formed by systems of dimension m {TZ"^), 
can be broken down in Q{Q — l)/2 subsystems of dimen- 
sionality 2m (7^^™). Then, the conditional exponents 
of the neural network between two subspaces measure 
the exponential divergence of nearby trajectories of the 
transformed equations for these two subspaces, which in 
practice is calculated using the following Jacobian in the 
method of Ref.J^] 



^^kl dxij / A N 



kl dx 



which is 



kl 



-5x11 -1 -5x^ 

4r -r 

3:,±_3£±^ + G' 3x11 -^(^^±^ 1 -1 

-5x^ -Sx" -1 

4r -r 



where x stands for Xki- For a network of Q fully 
connected neurons with equal coupling strengths, F = 
-QAki. When ^ = [(Q - 2)/2]4 - E?=i a'fc (with 
k 7^ is either orders of magnitude smaller than the 
quadratic terms (no synchronization) or ^=0 (CS), then 
G = 0. For parameter values close to or when PPS or PS 
is present, the quadratic terms are also small and G can- 
not be neglected. In Fig. [H this happens for the parame- 
ter region Aki,Aik = [0.15, 0.25]. There, Eq. (O seems to 
be violated. To resolve that, we force ^ = 0, which leads 
to G = 0, condition for which the Jacobian in Eq. (jAl[) 
can be used. We set the initial conditions all equal, and 



integrate the system for a small time interval (5 bursts 
which is equivalent to about 50 spikes) to estimate a fi- 
nite time averaged MIR, indicated in this figure by {Ic)f- 
Within the time scale for which phenomena happen in 
real biological neural networks, periodic state as well as 
chaotic state in the asymptotic sense might never be ob- 
served, but rather a transient state whose subspaces a;" 
and x-^ possess finite time conditional exponents. Finite 
time quantities are well defined in dynamical systems. 

For Q=2, F = -2Aki and G = 0. When CS takes 
place in a network formed by Q neurons, the only term 
of the Jacobian that changes is F. For this Jacobian, 
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we can calculate that CS appears if F{Q) < F{Q = 2), 
where F{Q = 2) = -2Aki{Q = 2), with A^iiQ = 2)=0.5 
being the couphng for which complete synchronization 
appears for a configuration of two coupled neurons [see 
also [11]]. So, the coupling to reach CS is Aki > l/Q, 
when the second largest Lyapunov as well as all transver- 
sal conditional exponents are negative. At this point, the 
trajectory distance between any pair of neurons tends to 
zero. 

For the active channels composed by coupled one- 
dimensional maps 

Q 

xl^l = 24^) + ^ 2c(4) - )), mod(l), 
1=1 

we can calculate the mutual information in each com- 
munication channel exactly, with no need of any special 
conditions. The network equations can be broken down 
in subspaces that depend only on the parallel or transver- 
sal variables of that subspace. So, a;|j'"^\^=i^xlf'-'^, and 

a^li'll^i" =G'a;l'^''"' + H{xi!^'''''^). H does not participate in 
the calculation of the conditional exponents and can be 
ignored. For a fully connected network formed by Q 
maps with equal couphng strengths c, F=2(l — Qc) and 
G = 2. The conditional exponents are A-'-=ln(]F]) and 
All = ln(|G|). For a network of 0=4 maps bidirection- 
ally connected to their nearest neighbors forming a closed 
ring, i.e. x^*' is connected to a;^*+^) and to x'*"^-', and 
x^^) is connected to x^^-^) and to x'^^\ then i^=2[l -2c] 
(F=2[l— c]) and G=2 for subspaces whose pair of systems 
have a direct connection (no connection). This network 
completely synchronizes when all the A^*- <0, and thus, 
when c > 1/2. 

APPENDIX B: INFORMATION DIMENSION, 
LYAPUNOV EXPONENTS AND MIR 

We consider an active channel formed by only one com- 
munication channel, composed by two coupled unidimen- 
sional systems which produce an attractor F with at most 
two positive Lyapunov exponents. F is corse-grained 
with volumes of size e [l3], and for e ^ 0, we have that 
P/ InP// In (e) = Di, with Di being the information 
dimension of F, a quantity that measures the informa- 
tion content of F and P/ is the probability of finding a 
trajectory point in one of the i volumes of size e. The 
average probability (P) of finding a trajectory following 
an itinerary visiting one of the possible m combinations 
of sequences of L volumes of size e for a time interval 

Lt is (P) cx cxp"''-^^.^ , and D^{''> -D?^ = 

and d[^^ G [0,1]) are the partial information dimensions 
(40| . a quantity that measures the information content 
of F along the direction j, either parallel or orthogo- 
nal to the trajectory, and Xj are the Lyapunov expo- 
nents in the direction j. •is the inner product. Assum- 
ing that the distribution of trajectory points is smooth 



along unstable directions (associated with positive ex- 
ponents) and the system possesses a Sinai-Ruelle-Bowen 

(SRB) measure [13, d[^^ ->1 if A^ > 0. From [H,!!!], 
l/(Ti)Em^ilnPi = Hks for SRB systems. To un- 
derstand how that is derived, we assume uniformity in 
the probability distribution, —J^m-^i^'^-Pi = — ln(P). 
Then, the term H^^f^ [in Eq. §1] can be calculated by 
knowing that — In (P)/(rL) ~ ^j- Now, we make 
the intuitive hypothesis that the terms cr^"^ and in 
Eq. preserve the physical quantities used to calcu- 
late a^°''^\ So, if CT^""''-' is a fimction of the information 
dimension and the Lyapunov exponents of the trajectory 
on the subspaces {a, (3), similarly, cr^") and ct'''' should 
be a function of these quantities. This hypothesis pro- 
vides the terms i:>^"^|A(")| + d['^^\X^'^^\ in Eq. ©. If 
a projection of F onto the lower-dimensional subspaces 
a and /3 produces a fractal set, one has to consider the 
absolute value of the negative Lyapunov exponent [2^ . 

Otherwise, d[°'^ — d[^^ = max [d[^^], assuming that the 
subspaces a and /3 contain only the dynamics of the ex- 
panding directions, i.e., either they are unidimensional 
or they can be reduced to a unidimensional subspace by 
a mapping of the flow [l8| . For most of the chaotic chan- 
nels here studied, max [D^"'^]=l, for typical projections 
of F. Further, we calculate the Lyapunov exponents of 
the transmitter and the receiver as if they were detached 
from each other. If the elements that compose the chan- 
nel have equal parameters, A^"-' = A*^'^-'=Ai=All , other- 
wise, A(") ^ X^i^\ 

For the considered networks, each pair of elements, in 
the transformed variables of the parallel and transversal 
subspaces, forms a composed subspace (a^, Pi) that pro- 
duces at most two positive conditional exponents. The 
previous analysis applies for each communication channel 
of this network, since each pair of neurons has a dynamics 
equivalent to a bidimensional discrete map with at most 
two positive conditional Lyapunov exponents, a larger 
one a||,; and a smaller one A^;. Then, 

Ic{a,,(3i) = xl-Xii, (Bl) 

if A^; > 0, or Ic{ak,(3i) = a|;, otherwise. To de- 
rive Eq. dm]), we have used that A("'=)=A(''')= a|!;, 

= max(D^^'^)=l, and H^^^'^'^ is the sum of all 
positive conditional exponents of the composed subspace 
(afc ,/?;). This can be done whenever the subspace {ak,(3i) 
is separable from the whole network. If the elements 
forming the channel have different parameters and one 
still wants to use Eq. (jBip . be in mind that such an 
equation might provide an upper bound for the MIR be- 
tween these two subspaces. 

For arbitrary networks, for simplicity let us imagine 
two coupled oscillators x^"^ and x'^^^ , coupled by a term 
c, the terms A*^"^ and A^''^ from Eq. ^ can be ana- 
lytically or semi-analytically calculated if the coupling is 
either sufficiently small (such that the elements forming 
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the network are almost decoupled) or sufRciently large 
(such that the whole network has a high level of syn- 
chrony). At this situation, A^"-* i^'"^^) is the Lyapunov 
exponent in the subspace of the oscillator x''"'> (2^*-''^), i-C, 
assume x^^'' = (af^"^ — 0) and c=0, and then calculate 
the Lyapunov exponents by the usual methods. For such 
coupling strengths, this is an equivalent approach to the 
one described in Sec. IVIII For other coupling strengths, 
we expect that i?[")|A(") | ^|A(«| - ffj,"/^ < H^^f. 
If not, that points to the existence of trajectory foldings 
in the lower-dimensional subspaces a and /? which results 
in an overestimation of the Lyapunov exponents, A*^"^ and 
A^'^). In such cases, Z?^ ' should be underestimated [4ll |. 
so balancing the action of the trajectory foldings. 

If the trajectory is very close to the synchronization 



manifold and so, Xi = X2 



xq, the term dxf.i/d: 



in Eq. (jAip gives the largest Lyapunov exponent A of the 
network which equals the largest exponent of one neuron, 
and thus, Eq. (|B1[) can be estimated by Ic{a.k,oti) — 
A — A^;. This equation agrees with the intuitive idea 
that the amount of information exchanged between two 
systems within a large network is given by the amount 
of information production of one system (A) minus the 
error in the transmission between both systems (A^^). 



APPENDIX C: PHASE AND PHASE 
SYNCHRONIZATION 

Phase synchronization [t*] is a phenomenon defined by 

l^fc -TO0(| <r, (CI) 

where (pk and 4>i are the phases of two neurons and ai , 
m = uii/ujk is a real number pi3 |. and cok and uii are the 
average frequencies of oscillation of the neurons and 
ai, and r is a finite number [43l |. In this work, we have 
used in Eq. (|C1[) m = 1, which means that we search 
for Uk ■ a;i=l:l (rational) phase synchronization Jj^. If 
another t ype of tuk ■ ^i-PS is present, the methods in 
Refs. [Mill 1131 can detect. 

The phase is a function constructed on a 2D sub- 
space, whose trajectory projection has proper rotation, 
i.e, it rotates around a well defined center of rotation. So, 
the phase is a function of a subspace. Usually, a good 2D 
subspace of the HR neurons is formed by the variables 
X and y, and whenever there is proper rotation in this 
subspace the phase can be calculated by [11] 



yx — xy 
(a;2 -I- y2) 



dt. 



(C2) 



If there is no proper rotation in the subspace {x,y) one 
can still find proper rotation in the velocity subspace 
(i, y) and a phase can be defined by Q 



yx — xy 
(i^ -I- y^) 



dt. 



(C3) 



If a good 2D subspace can be found, one can also define 
a phase by means of Hilbert transformation, which ba- 
sically transforms an oscillatory scalar signal into a two 
components signal [45|. In the active channel of Eq. 
for the coupling strength interval Aki = [0, 0.05], the sub- 
space (x, y) has proper rotation, and therefore, phase is 
well defined and can be calculated by Eq. (IC2p . How- 
ever, for this coupling interval, Eq. (jCip is not satisfied, 
and therefore, there is no PS between any pair of neurons 
in the subspace {x,y). 

For the coupHng strength interval Aki = [0.05,0.23], 
the neurons trajectories lose proper rotation both in the 
subspaces (x,y) and {x,y). The phase cannot be calcu- 
lated by Eq. JCSJ or by Eq. JCll. That is due to the 
fact that the chaotic trajectory gets arbitrarily close to 
the neighborhood of the equilibrium point (x, y)={0, 0), a 
manifestation that a homoclinic orbit to this point exists. 

In fact, the Hilbert transformation also fails to provide 
the phase from either scalar signals x or y, since these 
signals do not present any longer an oscillatory behavior 
close to the equilibrium point. In such cases, even the 
traditional technique to detect PS by defining the phase 
as a function that grows 27r, whenever a trajectory com- 
ponent crosses a threshold cannot be used. Since the tra- 
jectory comes arbitrarily close to the equilibrium point, 
no threshold can be defined such that the phase differ- 
ence between pairs of neurons is bounded. Notice that 
by this definition the phase difference equals 2ttAN. For 
that reason. Fig. 3)3 would remain roughly as it is even if 
the thresholds that define a spike and a burst are modi- 
fied or even if another variable (either y or z) is used. In 
this figure, a burst (spike) in a neuron ak is considered to 
start/end if Xk crosses the threshold defined by Xfe=-1.0 
(xfe^O.O). 

In order to check if PS indeed exists in at least one sub- 
space, alternative methods of detection can be employed 
as proposed in Refs. [H, [13]. In short, if PS exists in 
a subspace then by observing one neuron trajectory at 
the time the other bursts or spikes (or any typical event), 
there exists at least one special curve, F, in this subspace, 
for which the points obtained from these conditional ob- 
servations do not visit the neighborhood of F. A curve 
F is defined in the following way. Given a point xq in 
the attractor projected on the subspace of one neuron 
where the phase is defined, F is the union of all points 
for which the phase, calculated from this initial point xq 
reaches n{r), with n=l,2,3,...,c» and (r) a constant, 
usually 2tt. So, note that an infinite number of curves 
F can be defined. For coupled systems with sufficiently 
close parameters that present in some subspace proper 
rotation, if the points obtained from the conditional ob- 
servations do not visit the whole attractor projection on 
this subspace, one can always find a curve F that is far 
away from the conditional observations. Therefore, for 
such cases, to state the existence of PS one just has to 
check if the conditional observations are localized with re- 
spect to the attractor projection on the subspace where 
the phase is calculated. 
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FIG. 5: [Color online] The network of Eqs. Q for Ahi=Q.l. 
The curve F, a continuous curve transversal to the trajec- 
tory, is pictorially represented by the straight line F. a, the 
conditional observations are not localized and thus there is 
no PS in this subspace. The light gray line (green online) 
represents the attractor projection on the subspace (x, y) 
of the neuron a2, and filled gray circles (red online) repre- 
sent the points obtained from the conditional observations of 
the neuron 02 whenever the neuron 04 spikes. The point 
[x, y) = (0.0) does not belong to F. b, the conditional ob- 
servations are localized and thus there is PS in this subspace. 
Light gray dots (green online) represent the reconstructed at- 
tractor Z2{t) X Z2(t — T), for T—30, and filled circles (red online) 
represent the points obtained from the conditional observa- 
tion of neuron 02, whenever the reconstructed trajectory of 
the neuron 04 crosses the threshold line 24 (t — t) = 3.25 and 
04 (i) > 3. 



Conditional observations of the neuron trajectory ak in 
the subspace {x,y), whenever another neuron ai spikes, 
in the system modeled by Eqs. (O, are not localized 
with respect to a curve F, for the couphng strength 
0.05 < Aki < 0.23. An example can be seen in Fig. 
[5^, for Aki =0.1. The set of points produced by the con- 
ditional observations are represented by dark gray circles 
(red online), and the attractor by the light gray points 
(green online). Therefore, there is no PS in the subspace 
(x, y). However, the points obtained from the conditional 
observations do not visit the whole attractor in the sub- 
space (x, y). This is an evidence that there is PS in some 
other subspace. 

In order to know on which subspace PS occurs, we 
proceed in the following way. We reconstruct the neuron 
attractors by means of the time-delay technique, using 
the variable z. This variable describes the slow time- 
scale, responsible for the occurrence of bursts. The re- 
constructed attractor z(t) x z{t — r) has proper rotation 
(see Fig. [SJd) and the points obtained from the condi- 



tional observations do not visit the neighborhood of a 
curve F, then, there is PS in this subspace. Indeed, we 
find localized sets with respect to a curve F in the system 
of Eqs. in the reconstructed subspace {z{t)xz{t—T)), 
for Am > 0.1. 

So, for the parameter interval Aki — [0.1, 23[, there 
is no PS in the subspace (x, y) but there is PS in the 
subspace of the variable z. In this type of synchronous 
behavior, the bursts are phase synchronized while the 
spikes are not. This behavior is regarded as bursting 
phase synchronization (BPS). 

For simplicity in the analysis, we say that BPS happens 
when at least one pair of neurons is phase synchronous 
in the bursts. Partial phase synchronization (PPS) hap- 
pens in the network when it is true that for at least one 
pair of neurons Eq. (jCl|) is satisfied by the phases as 
defined by either Eq. or Eq. dUS]). In addition, at 

the coupling strengths for which PPS appears, one pos- 
itive Lyapunov and one positive transversal conditional 
exponent become negative. At the coupling strengths for 
which Eq. (jCip is satisfied for all pair of neurons (there 
is PS), the second largest Lyapunov exponent and all the 
transversal conditional exponents become non positive. 

Notice that these phenomena happen in a hierarchical 
way organized by the " intensity" of synchronization. The 
presence of a stronger type of synchronization implies in 
the presence of other softer types of synchronization in 
the following order: CS ^ PS ^ PPS ^ BPS. 



APPENDIX D: RECOVERY OF INFORMATION 

Equations ^ and (jBip give the amount of information 
that can be retrieved in the receiver per time unit. Imag- 
ine the neural network. If, in average, a burst happens 
for a time interval AT, one can retrieve in the receiver an 
amount /cAT of information about the transmitter per 
burst. It is often desirable to known how much informa- 
tion one single observation with precision e can provide. 
Assuming that observations are taken over in time in- 
tervals not smaller than St = — ln(e)/All, the maximal 
amount of information, /„, that can be retrieved in the 
receiver about the transmitter in each observation is esti- 
mated by Im — (A-'^/A'I — 1) In (e). We arrive at this result 
by assuming St to be the memory time of the channel, 
the time interval for which observations in the receiver 
trajectory with precision e, at a time to, will provide no 
information about the transmitter trajectory, at the time 
to -|- St, and 1^ = Ic ^ St. 



APPENDIX E: TRANSIENT DYNAMICS 

If an active channel is being externally stimulated or 
if the initial conditions are far away from the asymptotic 
(for large time intervals) stable state (periodic or chaotic 
behavior), the channel will present a transient dynamics. 
In such a case, Eqs. ([3]) and (jBip should remain valid by 
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the use of finite time conditional or Lyapunov exponents 
(assuming I?^ = l). As an illustrative example, an active 
channel that has an asymptotic chaotic attractor might 
have to be treated as a periodic channel (space contract- 
ing dynamics), if the initial conditions are far away from 
the chaotic set and the dynamics is dominated by the 
stable directions. 

Acknovifledgment We thank R. Kliegl who has, by a 
series of discussions, made us aware of the works in ex- 



[1] S. P. Strong, R. Koberle, R. R. de Ruyter van Steveninck 
and W. Bialek, Phys. Rev. Lett. 80, 197 (1998). 

[2] M. Kihara, O. Sadayasy, and P. Eskelinen, Digital Clocks 
for synchronization and communication (Artech Hause, 
2002). 

[3] C. Von der Malsburg, Ber Bunsenges Phys. Chem. 89, 
703 (1985). 

[4] W. J. Freeman, Clin. Beurophysiol. 44, 586 (1978). 

[5] W. Singer, Neuron 24, 49-65 (1999). 

[6] A. Treisman, Curr. Opin. Neurobiol. 6, 171 (1996). 

[7] A. Pikovsky, M. Rosenblum, and J. Kurths, Synchromza- 
tion A Universal Concept in Nonlinear Sciences (Cam- 
bridge, London 2003). 

[8] S. Strogatz, Sync: The Emerging Science of Spontaneous 
Order (Hyperium, New York 2003) . 

[9] G. Pareti and A. Palma, Neurol. Sci. 25, 41 (2004). 
[10] M. Shadlen and J. A. Movshon, Neuron 24, 67 (1999). 
[11] C. E. Shannon and W. Weaver, The Mathematical The- 
ory of Communication (The University of Illinois Press, 
1949). 

[12] J.-P. Eckmann and D. Ruelle, Rev. Mod. Phys. 57, 617 
(1985). 

[13] In order to be able to proceed with analytical calcula- 
tions, we only consider autonomous networks. However, 
note that every element of an active channel generates an 
information signal that can be interpreted as an external 
time-dependent input of the channel. 

[14] If the subspace a has euclidian dimension Da, and the 
subspace {3 euclidian dimension D^g, the subspace a, (5 
will have an euclidian dimension Da -f- Dp. The corse 
graining is realized by hypercubes with volume , e^i^ , 
and for the subspaces a, {3, and a,/3, respec- 

tively. At this particular point of the manuscript, we con- 
sider the active channel to be formed by only one receiver 
and one transmitter such that the subspace a, (5 repre- 
sents the whole channel. 

[15] A. Arnold, Ergodic Problems of Classical Mechanics 
(Addilson- Wesley, 1988). 

[16] Ya. B. Pesin, Russian Math. Surveys 32, 55 (1977). 

[17] F. Takens, Detecting Strange Attractors in Turbulence, 
in Dynamical Systems and Turbulence, Ed. D. A. Rand 
and L. S. Young (Springer- Verlag, New York, 1980). 

[18] In Eq. (|3]), we assume that the subspaces a and (5 con- 
tribute each with at most one positive (or one negative) 
A. Otherwise, a summation would have to be carried out 
over all possible positive (negative) exponents. 

[19] E. Bollt, Y. C. Lai, and C. Grebogi, Phys. Rev. Lett. 79, 
3787 (1997). 

[20] M. S. Baptista and J. Kurths, Phys. Rev. E 72, 045202R 
(2005). 



perimental psychology about the capacity limit of short- 
term memory, D. Gauthier who has convinced us of the 
importance to understanding how signals can be trans- 
mitted through channels that are space contracting, H. 
Kantz for the conceptual discussions concerning Eq. ([3]), 
K. Josic, and T. Nishikawa for another important series 
of discussions, and T. Pereira for a critical reading of 
Appendix C. 



[21] Systems of bidirectional equal couplings can be consid- 
ered as models of electrical gap junctions, a coupling that 
allows bidirectional flowing of information in neural net- 
works. 

[22] L. M. Pecora and T. L. Carrol, Phys. Rev. Lett. 64, 821 
(1990). 

[23] R. V. Mendes, Phys. Lett. A 248, 167 (1998). 
[24] R. Badii, G. Broggi, B. Derighetti, et al. Phys. Rev. Lett. 
60, 979 (1988). 

[25] E. Ott, Chaos in dynamical systems (Cambridge Univer- 
sity Press, New York, 2002). 

[26] F. Ledrappier and L. S. Young, Ann. Math. 122, 508 
(1985). 

[27] In fact, D^^'a'"^ = |A'''> | [H, which means that the 
rate of information produced in the transmiter dJ'^' A'-"'' 
is conserved by the channel, and thus completely arrives 
at the receiver. 

[28] M. S. Baptista and L. Lopez, Phys. Rev. E 65, 055201R 
(2002). 

[29] R. R. Llinas, Handbook of Physiology; The Neurvous 
System (Am. Physiol. Soc, Bethesda, 1981); S. P. Mar- 
shall and E. J. Lang, J. Neurosci. 24, 11356 (2004). 

[30] M. C. Eguia, M. I. Rabinovich, and H. D. I. Abarbanel, 
Phys. Rev. E 62, 7111 (2000). 

[31] F. T. Arecchi, Lecture notes in computer science 3146, 
90 (2004). 

[32] M. S. Baptista, C. Grebogi, and R. Koberle, Philo- 
sophical Transactions of the Royal Society, 366 (2008), 
[doi:10.1098/rsta.2007.2093]. 

[33] J. L. Hindmarsh and R. M. Rose, Proc. R. Soc. London 
B 221, 87 (1984). 

[34] R. Luna, A. Hernandez, C. D. Brody, et al, Nature Neu- 
roscience, 8, 1210 (2005). 

[35] If we had defined Cc as the maximal of the MIR for 
all possible parameter realizations in the elements of the 
channel, we would have concluded that the new defined 
capacity would be always reached for channels composed 
of elements that have equal parameters. Since, we are 
often dealing with channels composed of elements that 
possess either equal or very similar parameters, the many 
channel capacities calculated throughout the paper can 
be considered to be the maximum over many coupling 
and parameter configurations. 

[36] For A2 < 0, one has to take into account in the calculation 
of Ic, using Eq. Q, the existence of a fractal set. Notice 
however that Eq. (jBlJ can always be used to estimate 
the MIR, even if A2 < 0. From that, we arrive at the 
same conclusions concerning the channel capacity, i.e., 
Cc(e = 0) >Cc(e>0). 



15 



[37] D. J. Watts and S. H. Strogatz, Nature 393, 440 (1998). 

[38] N. Schweighofor, K. Doya, H. Fukai, et al. Proc. National 
Academy ol Science, 101, 4655 (2004). 

[39] I. Belykh, E. de Lange, and M. Hasler, Phys. Rev. Lett. 
94, 188101 (2005). 

[40] R. Badii and A. Politi, Complexity, Hierarchical struc- 
tures and scaling m physics (Cambridge, UK, 1997). 

[41] W. Ott, B. Hunt, V. Kaloshin, Ergodic Theory and Dy- 
namics! Systems, 26, 869 (2006). 

[42] M. S. Baptista, S. Boccaletti, K. Josic, and I. Leyva, 
Phys. Rev. E, 69, 056228 (2004). 



[43] M. S. Baptista, T. Pereira, and J. Kurths, Physica D, 

216, 260 (2006). 
[44] T. Pereira, M. S. Baptista, and J. Kurths, Phys. Lett. A, 

362, 159 (2007). 
[45] D. Gabor, J. IEEE London 93, 429 (1946). 
[46] M. S. Baptista, T. Pereira, J. C. Sartorelli, et al., Physica 

D 212, 216 (2005). 
[47] T. Pereira, M. S. Baptista, and J. Kurths, Phys. Rev. E, 

75, 026216 (2007). 



